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ABSTRACT 

This paper outlines the procedures used to implement 
and test the CARL version ClOO circulation system at the University 
of Colorado's Boulder Campus. Intended to serve as a guide to other 
libraries implementing new circulation software, the paper describes 
a broad approach to testing such software, describes procedures that 
speed and organize the process, and suggests staffing levels and 
equipment to have available. The focus is on describing methods to 
test the accuracy of data transfer, determine the functionality of 
the features, and to confirm that the local parameters interact 
properly with all circulation functions. The following steps are 
covered: (1) test preparations, such as involving technical staff and 
organizing work area and materials; (2) documenting the transfer of 
data from the old system to CARL by preparing forms for checking the 
accuracy o^ the transfer and providing a means, such as fax or email, 
to document discussions with CARL about problems identified during 
the test; (3) implementing test procedures to check the accuracy of 
the transfer of data on materials, patron records, and transactions; 
(4) testing to make sure functions and features, such as In-transit, 
Added Borrower, and Renew-All, perform as they are supposed to; (5) 
stress testing the system by such means as adding 500 books to a 
patron's account and renewing them all; (6) alert CARL that the 
system is ready to go live after identified problems are cleared up; 
and (7) continuing to retest critical elements such as parameters and 
functions after the system goes live. The prospective user of these 
guidelines is warned that although testing is time and labor 
intensive, it is essential; even though others have implemented the 
system, there may still be bugs that have not been resolved. (KRN) 
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My discussion addresses tfie testing of CARL'S new circulation at your site. My 
intent is to 



□ outline a broad approach to organize the process, 

□ describe some procedures you may want to consider, and, 

□ suggest some documentation you may want to have on hand. 



IMPLEMENTATION SEQUENCE 



The entire implementation sequence is straightf onward: 



□ Holdings, patron, and transaction files are duplicated, 
converted, and loaded into a test system along with your 
parameters and displayed on a specially designated terminal 
you select. 

□ Test for the: 

accuracy of data transfer, 
^functionality of each feature, 
^ interaction between parameters and 

functions. 



□ With all corrections made to the test system, the live files are 
frozen and converted. The corrected software is loaded and 
brought up live. 
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□ Once the new system is running live at the circulation desks, 
you need to retest the live files again for: 
p» accuracy of data transfer, 

functionality of each feature, 
p» if parameters correctly interact with -PERMISSION TO REPRODUCE THIS 

functions material has been granted by 
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TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



Essentially, HI talk about that second step in detail, tliat is: verifying the accuracy of 
the the data transfer, functionality of features, and parameter interaction. 

The purpose of testing is to: 

□ confirm that records were converted properly, 

□ determine that the functions operate as expected, and, 

□ establish that the software is interacting with the parameters 
correctly. 

For example, you have to be confident that a patron who charged seven books on 
the cid system still has seven charged after the new system is installed. You'll need 
to be confident that the parameters work properly in every instance. You need to 
confirm that the functions work as they are described in the documentation. 

Broadly, you'll test for two things: 

□ accuracy of data transfer, and, 

□ functionality of the features. 

For example, did the media codes, due dates, fines, patron addresses, and so on. 
transfer correctly? Do the functions— Renew- All, In-transit, fiscals, etc.— actually 
work? The objective is to confirm such things, identify the problems, and have 
most problems resolved in time to implement the new system according to the 
schedule you had established. 
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PREPARING FOR TESTING 



Communication & Personnel 

Long before your test files are created, there are certain things you can prepare that 
will make the testing process much more efficient. 

There are individuals, both inside and outside of your department, that need to be 
made aware of what is happening and the implications. There is quite a bit of 
documentation that can be prepared in advance. There is equipment that needs to 
be set aside. Staff time to the effort can be assigned. 

As early as possible begin talking with the people who will be directly and indirectly 
involved with the conversion and develop a means to keep them informed. Don't 
underestimate how large a group of people that can be. 

For example, Boulder has six circulation desks scattered all across campus. There 
were about 20 technical level staff who had a real stake in the outcome of the testing 
process but were not active in process. Unless such people are involved or kept 
informed the rumors can quickly overtake reality. Circulation staff need to see the 
system early on— perhaps through an early training session— then kept informed of 
progress as part of regular circulation meetings, or with memos, or through electronic 
mail. 

There are also people outside of circulation that need to be involved. The Holdings 
file will be converted, and those records will be different. Consequently, there will 
be some technical services staff who need to be kept informed of progress. They 
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will need to be involved with the scheduling of final implementation, since it will bring 
down the Holdings file. 

Remember, things can go wrong that impact those departments. For example, 
when Boulder's circulation went live our Bibliographic Maintenance Unit couldn't write 
holdings records for days. 

Documentation 

In addition to drawing people into the process and making sure that key individuals 
are infomied, there is some documentation you can prepare prior to bringing up the 
test system. And by preparing this in advance, you'll make much more efficient use 
of your testing time. 

The most obvious pieces of documentation you should have on hand are your 
revised parameters and copies of the new circulation manual. 

You may also consider preparing an outline of the various system functions and who 
will be responsible for testing them. You may want one person to test your fiscals, 
another to test conversion, and someone else to check parameters. There are real 
advantages to testing that way, but it's easy to lose control of the process when it's 
split between several staff. A document outlining who is to check what will help 
direct the entire process. Anne Harmon and Steve Wrede developed a very good 
outline based upon the UNC conversion. You may want to adapt that. 

About a week before testing, we developed some charts to help manage the 
detailed checking. For example, this particular one has borrower types on the side 
with all possible parameters across the top— number of renewals allowed, 
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maximum number of charges, eta The cells are empty. The chart can be given to a 
student assic ant. By charging and renewing books at the test terminal, they simply 
fill in the blanks based on what the system is doing. 



I think this method has a couple of advantages. First, it regulates a tedious process 
to someone else. More importantly, the detail is being checked by someone who 
doesn't know the answers in advance. Consequently, they probably don't have the 
expectations and assumptions as someone who works with the system may have. 
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Checking for lost charges and fines can be made easier by the same process. At 
Boulder, there are about a dozen different time frames in which an item can become 
overdue or go iost mostly depending upon the media code but also on the 
borrower type. Additionally, some of our borrower types don't get fined, some get 
blocked and charged for lost books, and others just get blocked. Confirming all the 
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possible interactions between Borrower types and media codes can be very 
confusing. 



Borrower Type Patron ID Media Code Item Number $ Fine $ Lost Block 
RRCIP 123-45-6789 



ID 


U18301 


2563983 


IX 


U18301 


8763209 


2D 


U18301 


3987127 


::x 


U 18301 


6788903 


:^D 


U 18301 
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:\x 
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7D 
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3456629 
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3209432 


RE 
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2W 
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MR 
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MX 
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LO 
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Blank 


U18301 
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UCFS 



234-56-7890 



ID 
IX 
2D 
2X 
3D 
3X 
7D 
72 



U18301 
U18301 
U18301 
U 18301 
U18301 
U18301 
U18301 



6778903 
8842229 
5409832 
3098629 
7659432 
2167801 
8344625 



This sheet has 7 columns: Borrower Type, Patron ID, Media Code, Item Number, 
$Fine, $Lost, and Block. So, for example, Borrower Type called RECIP with this ID 
has each of these item numbers charged to it. Each item represents one of all 
Boulder's possible media codes. 
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One copy of the list goes to CARL; keep another for yourself. CARL will cause all 
the books to go lost. Using this list and your parameters report, you can determine If 
the items are fined, billed lost, and blocked appropriately. As you do this several 
times and check off the results, any problems will appear as patterns. 

So, review your parameters prior to testing and think of ways in which they interact. 
Any time they interact, you can design a table for testing. Your testing time will be 
much more productive and organized. 

Printouts 

In addition to the parameters, the manual, an outline, and some tables, you'll need 
printouts. Just before your test system is brought up, print out examples of: 

□ pati.^n records, 

□ holdings re;:ords, and 

□ detail screens from Patron Inquiry of books charged to various accounts 
and any fines assessed. 

Assign a student to make as many printouts as they can— if possible a couple 
hundred each. 

Materials 

In addition to all the documentation, you'll need books on hand to be able to use 
during testing. Have several hundred ready. You'll probably want to charge the 
books to a special code on the old circulation system so they will display as 
"checked out' on PAC. That way patrons can place a Hold or a Recall on the item. 
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The reason for this many items will become apparent when I talk about the testing 
process. 

Equipment 

Finally, you're going to need some equipment. You'll need at least one CARL 
terminal, however, I recommend two. These will be used to display the test files 
once they are available. Each should have light wands and printers. Make sure a 
telephone is nearby. Also, you'll want to have all those books you charged out next 
to the terminal. Shelving or booktrucks are going to be necessary. In a perfect 
world, you would have a fax beside you, too. But wherever the fax is located, plan 
on using it heavily during testing. You may want to make arrangements in advance 
with whoever is responsible for that machine. 

Yc j may also want to consider having a terminal nearby through which you can send 
electronic mail. In our testing process we used email extensively. By sending an 
email message (not CARL email) you can copy in any individual or group of 
individuals while building a historical file of the process. 

Having a written record is extremely helpful. A telephone conversation is 
expedient, but after a couple of weeks you won't rememt>er if you really discussed 
a problem, what was resolved, or the resolution. We have well over 1 50 pages of 
email documentation of this process. I can't overemphasize how useful it is to be 
able to refer to a specific message sent on a specific day. 

Returning to the equipment: you'll want the test terminals to be located away from 
the circulation desk. You wont get anything done if you have constant interruptions. 
Also, you don't want anyone mistaking a test terminal for one running live circulation. 
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That outlines the documentaticn. equipment and personnel concerns you can 
prepare in advance. Between the parameters, manual, outline, grids and printouts, 
you'll end up with quite a stack of paper, but you'll use it all. Preparing in advance wilt 
save a lot of labor later and will make the testing much more efficient. And this 
process needs to be efficient. Don't underestimate the amount of time or labor this 
will take. 

TESTING PROCEDURES 

Now, at some point, you'll have a test system running on its own terminal and a 
mound of documentation in a remote room someplace. The test terminals are 
independent of the live system running at the front desk. They operate in a universe 
of there own. If, for example, you look at one of the books charged out earlier for 
testing it will say "checked out" in the live system. In the test system, however, you 
can take the same book, discharge it, charge it to someone else, cause it to go lost, 
change the media code, recall it, and return it. If you look back at the live system, the 
record hasn't been touched; it's still charged out to the same code. Since the test 
terminals operate independently of the live system, you have a great deal of 
flexibility to experiment, and you don't even have to clean up after yourself. 

Believe it or not, there is a fairly logical sequence in which to test the software: 

□ determine the accuracy of the data transfer for the patron, 
transaction, ani the holdings files, 

□ confirm that that the functions, such as Charge, Renew, In- 
transit, and so on work as intended. 
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□ verify that parameters are loaded correctly. This is to 
ensure that loan periods are correct, that fines calculate 
correctly, blocks work correctly, and so on, 

□ perform as best a stress test as you can. Make sure you 
really can do a renew-all on an account with 700 books 
charged to it, that you can manipulate a bindery account 
with 1 ,400 titles attached to it, and so on. 

O.K., we'll go through these one by one. 

Files 

First, determine the accuracy of the data transfer. Start with the holdings file. 
Remember all the printouts of the Holdings Edit screens? This is when you use 
them. Record by record, compare the printouts to the test screens. Check the call 
numbers (particularly if you're transferring into buckets), check the titles, the unique 
Identifier, the media codes, and everything. The purpose is to make sure that all 
the fields transferred, the 3's didn't transfer as 8's, and so on. 

Record the discrepancies directly on the printout. If there are any problems, it won't 
take long for the patterns to emerge. Soon you'll have multiple examples of a 
particular problem along with specific records in which it occured. You can either fax 
the printouts to CARL, write an email note identifying the specific records, or discuss 
it over the phone. Whatever you do, don't throw the printouts away. Because at 
some point, CARL will report that the problem is fixed. You'll need to refer back to 
the printouts to find the specific records. Having the original notes in front of you is 
very useful, too. 

Use the same process to check the ^\ccuracy of the patron records. Using the 
printouts as masters, check that the ID numbers are correct, that the addresses are 
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properly placed and the address flags are property set, and so on. Again, any 
discrepancies are recorded on the printout, reviewed, reported to CARL, printouts 
filed, then pulled out again to review the fixes. 

Checking the transactions can be slightly more problematic than working with the 
holdings or patron records. Basically, you will check that titles have remained the 
same, due dates haven't changed, hold:: and recalls are still In place, blocks are 
present, fines transferred into the Archive correctly, and any notes transferred 
properly. But there could be last minute activity on the patron's account that app)ears 
as a conversion discrepancy. So you may have to backtrack to the live system to 
confirm the accuracy of some records. 

If all this sound tedious, it is. But having the printouts made in advance saves a lot of 
time during this phase: you don't have to run from a live terminal to a test terminal, the 
printout is a convenient form for recording discrepancies, and it provides a 
permanent record so that you can do follow up easily. 

Functions 

Once satisfied that the data files transferred accurately and that any problems are 
resolved, you can move on to testing functions. This is a more ambiguous process 
than checking the files for accuracy, and, to some degree, more serendipitous. 

The intent is to confirm that the functions and features perform as they are supposed 
to. Do not assume that thev will . Either because of faulty coco or because of 
peculiarities of CU Boulder libraries, we identified several functions and features that 
needed work. The In-transit, Added Borrower, and Renew-All functions, at one time 
or another, didn't work for us. 
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The circulation manual is a good guide to use in evaluating the functions. Literally, 
you can start on page one and work through charge, renew, return, and so on. At this 
point you are basically confirming that functions work as they should. You'll need to 
perform a few dozen transactions for every function. Check that the features interact 
properly with your records. Make sure that the bridges work. 

There's a lot of stuff to check, and don't just check the new features. Check 
everything. Just because a function worked in a certain way in the old system 
doesn't mean it will work the same in the new. Many of us have installed unique, 
enhancements to make the system perform in a particular way. Those old 
enhancements may not be in the new system. So you want to identify those 
absent enhancements that are no longer layered into the software and determine if 
you still need them or not. 

Once satisfied that the functions and features are sound, you can begin testing 
parameters. The tables that I described earlier really speeds this process. Simply 
have someone fill in the blanks based upon the sy^- v ti results and compare the 
completed grid with a master copy. Double-check ti ic discrepancies and report any 
problems. At some point CARL will report back that they are fixed. Go back and 
check them again. Our experience was that parameters would somehow slip. One 
day Uiings would function correctly and then not the next. So plan to check the 
parameters several times. 

Checking the functions and observing how features interact with various types of 
records is a particularly long process. You have to do a lot of transactions with a lot 
of different records to identify patterns. And it's often difficult to stay focused since 
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one thing leads to another. You may see the results of a problem while looking at 
Inquiry but, in the process of tracking the cause, end up in Holdings Edit. Meanwhile, 
you'll have questions about two other things along the way. 

Documenl suspected problems as carefully as possible. Describe the problem 
clearly and always submit multiple examples of what you report. We found that the 
more analysis we did on a problem before reporting it, the more likely the problem 
would be resolved quickly. 

So, at this point you've: 

□ confirmed the accuracy of the records, 

□ tested functions, and, 

□ confirmed that the parameters are set correctly, and that 
there are no unexpected interactions between the 
parameters and the features. 

You'll have identified a lot of problems and, hopefully, have them mostly resolved. 
You'll also have a good idea about what is outstanding and how well the system is 
functioning as a whole. 

Since about 80% of the testing is completed, this point would be a good time to 
prioritize the remaining outstanding problems. Then communicate to CARL how 
much of that list must be functioning before you will agree to go live. Be sure to get 
the list to them in a reasonable amount of time. 

The final thing you want to do is stress testing. Now, you won't be able to do a true 
stress test. But, you can have a good time trying to crash the system. 
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stress testing is the electronic equivalent of slash and burn. For example, find a 
patron's account with 400 or 500 books on it. Charge another 400 or 500 to it— all 
the books you have on hand for testing— then do a renew-all. Interrupt the renewal 
process three quarters of the way through. Go back and stack multiple recalls and 
holds on 200 or 300 of the books then renew them, and so on. Cause all 900 of 
them to go overdue at once. Make bizarre fine payments. Watch for things such as 
response time, accuracy in screen displays, and so on. 

As the dates for implementation are confirmed, you'll want to get some memos out 
explaining what's happening to the other departments. Naturally, you want to make 
sure the circulation desk staff know what to expect and you may want to consider last 
minute training. Finally, and this is important, once the system goes live be sure to 
retest critical things such as parameters and functions. 

OK, to sum up: 

□ make testing a priority, 

□ don't underestimate the time and labor involved, 

□ don't assume that because someone else has implemented 
this system that all the bugs are resolved, and, finally, 

□ identify key individuals and keep them informed. 
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ABSTRACT 



This paper, originally presented at the Second Annual CARL User's Group 
Meeting in September of 1992, outlines the implementation and testing procedures 
used at the University of Colorado's Boulder Campus in the implementation of the 
CARL version C100 circulation system. 

The paper outlines a broad approach to testing new circulation software, describes 
procedures that speed and organize the process, and suggests staffing levels and 
equipment to have available. The focus is on describing methods to test the 
accuracy of data transfer, determine the functionality of the features, and to conf irm 
that the local parameters interact properly with al! circulation functions. 
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